1. Field of the Invention
The present invention relates to a video noise reduction method, and more particularly, to a video noise reduction method using adaptive spatial and motion-compensation temporal filters.
2. Description of the Prior Art
In an era of multimedia communication, image data is playing an important role. But no images are absolutely perfect no matter how good a camera is, since images are interfered with by the presence of noise. The principal sources of noise in digital images arise during image acquisition (digitization) and/or transmission. The performance of imaging sensors is affected by a variety of factors, such as environmental conditions during image acquisition, and by the quality of the sensing elements themselves. For instance, in acquiring images with a CCD camera, luminosity and sensor temperature are major factors affecting the amount of noise in the generated images. Images are corrupted during transmission principally due to interference in channels used for transmission. For example, an image transmitted by a wireless network might be disturbed as a result of lightning or other atmospheric charged particles. In addition, due to a sensing element with one pixel consisting of a transistor, instability of the transistor will always cause the sensing element a permanent or transient failure. That is so-called transient noise and permanent noise in image sequences.
Image sequences are often corrupted by noise in different production ways, such as a bad reception of television pictures. In the recent years, a number of nonlinear techniques for video processing have been proposed, which are known as superior to linear techniques. These algorithms are necessary, since the presence of noise in an image sequence degrades both its visual quality, as well as, the effectiveness of subsequent processing tasks. For example, the coding efficiency obtained for a particular image sequence is decreased by the presence of noise. This entropy of the image sequence is increased obviously by the noise. Therefore, filtering methods for reducing noise are desired not only to improve the visual quality, but also to increase the performance of subsequent processing tasks such as coding, analysis, or interpretation.
As for discussing temporal noises, noises are divided into three categories which including permanent noise, temporary transient noise, and temporary intermediate noise. Their features are described as follows:
1. Permanent Noise:
This kind of noise arose from the failures of the image sensors. It is generally caused by flaws in the manufacture of transistors, so the permanent noise always appears at the same locations in images.
2. Temporal Transient Noise:
The characteristics of this kind of noise are that pixels are interfered by noise in some frames but are not interfered by noise in some frames. The main reason is due to interferences by external environment or due to instability of sensing elements.
3. Temporal Intermediate Noise:
The characteristics of this kind of noise are that pixels are interfered by noise in the current frame, but are not interfered by noise in the next frame. Then pixels are interfered by noise in the next following frames again. This action is run alternately. The main reason is caused by external environment factors.
Generally, temporal video noise reduction filters are divided into four primary kinds:
1. Non-Motion Compensated Spatiotemporal Filtering:
Many of the non-motion compensated spatiotemporal filtering approaches used in the present day are developed from well-known 2D filtering techniques.
2. Motion Compensated Spatiotemporal Filtering:
To take full advantage of the temporal correlations that exist in an image sequence, explicit motion estimation and compensation have been used in a separate step or simultaneously with the filtering of the image sequence. From a filter design point of view, the addition of motion compensation to a non-motion compensated filter does not result in a new filter. However, it does improve the temporal correlation, which improves performance of the filter.
3. Non-Motion Compensated Temporal Filtering:
Due to the lack of robust motion estimators, early attempts at temporally filtering image sequences were restricted to simple frame average techniques. Although the temporal correlations are considered in this method, but the possibility of object motions are not considered. Therefore, there are residual images or fuzzy phenomenon appeared when an object moves.
4. Motion Compensated Temporal Filtering:
The method can avoid problems associated with non-motion compensated temporal filtering.
In general, higher visibility thresholds will occur in either very dark regions or very bright regions of pictures, and lower visibility thresholds will occur in medium to dark-gray regions. It was found that human visual perception is sensitive to luminance contrast rather than absolute luminance values. The ability of human eyes to detect differences between an object and its background, known as contrast sensitivity, is dependent upon the average value of background luminance. According to Weber's law, if the luminance of a test stimulus is just noticeable from the surrounding luminance, the ratio of just noticeable luminance difference to stimulus luminance, known as Weber fraction, is approximately a constant. However, in a real situation, due to the presence of ambient illumination or TV monitor characteristics, noise in dark areas tends to be less visible than that in regions of high luminance. Therefore, the modification has been made that, as the background luminance is low, the Weber fraction increases as the background luminance decreases.
The just noticeable difference (JND) is the ability of human eyes to distinguish luminance variation. A JND model we quote is presented by Chou and is described by the following expressions:
            JND      ⁡              (                  g          ⁡                      (                          x              ,              y                        )                          )              =          {                                                                                    T                  0                                ×                                  (                                      1                    -                                                                                            g                          ⁡                                                      (                                                          x                              ,                              y                                                        )                                                                          127                                                              +                                          h                      1                                                        )                                            ,                                                                          g                ⁡                                  (                                      x                    ,                    y                                    )                                            ≤              127                                                                                                            γ                  ×                                      (                                                                  g                        ⁡                                                  (                                                      x                            ,                            y                                                    )                                                                    -                      127                                        )                                                  +                                  h                  2                                            ,                                            otherwise                              }        ;where g(x,y) denotes a gray-level value of a pixel located at a position (x,y), and JND(g(x,y)) is its JND value of the pixel. According to experiment data of our present invention, h1=23, h2=23, To=11 and γ=7/128 will be more suitable for human visual perception on a video.
Noise can be added to still images or video sequences in various steps such as image acquisition, recording, and transmission. Then the results of post-processing tasks are also influenced by noise. As a result, noise reduction is important for video processing. For video-noise filtering, it may cause object-overlapped phenomenon in an image frame due to the occlusion problem when the spatial-filtering is only used, excluding the temporal-filtering. Oppositely, the image may be blurred and even the noise cannot be filtered if the temporal-filtering is only performed but spatial characteristics of the image are not utilized. Therefore, for removing noise correctly, a video-noise filter should be able to work on both temporal and spatial domains.
Continuous pictures in temporal domain form a motion of humans or objects. This characteristic is used in motion compensated methods to reduce redundancy in temporal domain. As for the current pictures, image areas that are the same as the corresponding areas in the previous pictures will not be transmitted to decrease data transmission. Searching similar areas between different pictures is so-called “motion estimation”. Displacements used for representing motion degree are called “motion vector”.
Applications of reducing video noise are already disclosed by some scholars.
For example, two non-motion compensated spatiotemporal filters are provided, a Video α-trimmed mean filter and a K nearest neighbor image sequence filter. Therefore, there are residual images or fuzzy phenomenon appearing when an object moves due to motion estimation not being used to detecting whether an object is moving or not.
In another reference document, a temporal filter (an adaptive motion compensated frame averaging) utilizing motion compensation is provided. This method is based on blocks and switched between simple frame averaging and motion compensated frame averaging to reduce video noise, which is one kind of the motion compensated temporal filters. This may blur outline edges of image after filtering noise due to problems of spatial outline edges of image, permanent noise, and temporary transient noise are not considered.
In addition, a method of adding motion compensation into a non-motion compensated filter is provided. The method includes three steps to reduce video noise: spatial filtering, motion compensation, and temporal filtering. The method of work of the spatial filtering is keeping information of edges and details of each frame and using a data-dependent weighted average filter (DDWA) to accomplish this step. The spatial filtering is added due to noise interference affecting accuracy of motion estimation. However, the output of the front end cannot reduce noise sufficiently, Boyce's detector are used for motion compensation.
In another reference document, only spatial information is used to reduce video noise, noise in temporal domain cannot be reduced.
Thus it can be seen, there is residual image phenomenon appearing when objects move if only spatial filters are used to reduce noise without considering temporal correlations. There is temporary transient noise appearing in outline edges of image and outline edges of an image may be blurred, if only temporal filters are used to reduce noise without considering spatial correlation. In order to reduce noise correctly, a video filter has to work both in the spatial and temporal domains. A method provided in the prior art reference document considers both characteristics in the spatial and temporal domains, but is applied to reduce Gaussian noise only but not to reduce impulse noise effectively.